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Abstract 

From the sandpoint of neural network dynamics we consider dynamical 
system of special type pesesses gradient (symmetric) and Hamiltonian 
(antisymmetric) flows. The conditions when Hamiltonian flow properties 
are dominant in the system are considered. A simple Hamiltonian has 
been studied for establishing oscillatory patern conditions in system under 
consideration. 

keywords: Neural network, Hamiltonian dynamical systems, gradient 
dynamical system. 

1 Introduction and setting the problem. 

It is well known PQ that synaptic connections in biological neural networks are 
seldom symmetric since the signal sent by neurons along their axons are sharp 
spikes and the relevant information is not contained in the spikes themselves 
but in the so called firing rates, which depends on the magnitude of the mem- 
brane potentials which governs all the process. On the other hand it should 
be pointed out that the recent neurophysiological observation of extremely low 
firing rates without some doubt on the general usefulness of this notion as re- 
ally the relevant neural variable. Thereby one can use some natural continuous 
variables to describe neural networks as dynamical systems of special structure 
like gradient (symmetric) and Hamiltonian skew-symmetrical flows. This gives 
rise to making use of a lot of methods and techniques for studying the struc- 
tural stability of the networks and the existence of so called coherent temporal 
structures fitting for learning process. 

Based on the considerations above one can introduce a class of nonlinear 
dynamical systems 

du/dt = K{u) (1) 
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where M 9 u is some smooth finite-dimensional metrizible manifold K : M — > 
T{M) is a vector field on M, modeling the information transfer process in a 
biological neural network under regard. The question is what conditions should 
be involved on the flow Q for it to be represented as follows: 

K{u) = -gradV(u) -${u)VH(u), (2) 

that is as a mixed sum of a gradient flow and a Hamiltonian flow on M. Here 
V : M — » R is the potential function and H : M R is the Hamiltonian 
function relevant to the flow (JTJ, grad := <7 _1 (u)V, V := : it G Afj, g : 
T(M) x T(M) -> i?+ is a Riemannian metrics and i? : T*(M) -> T(M) is a 
Poisson structure on M. 

Thus we need to find the corresponding metrics and Poisson structure on 
M subject to which the representation J5J holds on M. We shell dwell on this 
topics in the proceeding chapter. 

2 Poissonian structure analysis 

Assume first that the representation J5J holds, that is 

-&(u)VH(u) = K{u) + grad V{u) := K v (u) (3) 

for all u £ M and some d and g structures on M. This means therefore that 
the constructed vector field 10 is exactly Hamiltonian. Thereby one has ([3]) 
the expression 

#- 1 (u) = tp'(u)-ip'*(u), (4) 

where (p £ T*(M) is some nonsymmetric solution to the linear determining 
equation 

dp/dt + K$cp = V£. (5) 
Here, by definition, the flow Ky is defined as 

du/dr = K v (u) (6) 

and C : M — > R is a suitable smooth function chosen for convenience when 
solving |0J. It is clear (see [3]) that the symplectic structure (0J doesn't depend 
on the choice of the function C : M — > R. 

As the second step, assume that the metrics and Poisson structures on M 
are given a priori. Then due to (0 the following equation for determining the 
potential function V : M — > R holds: 

<p' ■ K + iff ■ gradV^ + K'* ■ <p + (gradF)'* -cp = V£, (7) 

where the element ip £ T* (M) has been assumed also to be known a priori as a 
solution to the equation Q). The expression (0 is a linear second order equation 
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in partial derivatives on the potential function V : M — > R. If this equation is 
compatible, then its solution exists and the decomposition holds. 

As one can check, the equation (7J almost everywhere possesses a solution 
for the vector if) = grad V, that is the following expression 

gradF = < t /}=g- 1 VV (8) 

holds on M for some i/' G T(M). Thereby, one gets 

w = (9) 

Making use now of the well-known Volterra condition (see (VV)'* = (VV)', 
we obtain the following criterion on the metrics g : T(M) x T(M) — > i? + : 

(#)'* = (10) 
Since from © also one follows that 

(giP,u x ) = (VV,u x ) = dV/dx, (11) 
the condition (|llf) is evidently equivalent to such one: 

(giP)'*u x -±(gi>) = 0. (12) 
ax 

Calculating the left handside expression of 1)1 2|l one gets the following final 
result: 

(g'*u x - g'u x )ip = gip'u x - ip'(gu x ), (13) 

which is feasible at check, if the metrics is given. Otherwise, if this is not the 
case, the linear expression 1)1311 determines a suitable metrics as its solution 
subject to the mapping g : T(M) x T(M) — > R + . As soon as the equation 1)13)1 
is compatible, its solution exists defining a suitable metrics on the manifold M. 

The results delivered above can be successfully applied to many interesting 
dynamical systems modeling information processes in neural networks, men- 
tioned in introduction. Below we shall demonstrate some of them having appli- 
cations at studying coherent temporal structures. 

3 Coherent temporal structures formation. 

One considers a network with two groups of neuron {xi G R : i=l, n\ and {yj £ 
R: j=l,m}, connected in such a way, that inside both groups the synaptic 
strengths are symmetric, whereas between groups they are antisymmetric. That 
is, neurons {x} are excitatory to {y} and neurons {y} are inhibitory to {x} . 
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This model is expressed in the form J2J), where 



V = 



i=l ^ ' t,j'=l 

3=1 \ / i,i=l 



,(2) 



^ / n rn n m \ 

H = 2 E* 2 + E^ 2 + EE w ^% ( 15 ) 

\i=l j=l i=lj=l / 

with the standard metrics g = l,a skew-symmetric Poisson structure i) = J6 







(16) 



or 



with constant /3 and elements u>ij being parameters, Ir n ,m) = {^ij '■ i = l,n, j = 
l,m}, Ji n ) and Jt m ) being some skew-symmetric matrices. 

It is worth to mention here that the representation J2J with structures l|16fl 
is not unique and some other solutions to the equation (|13|) can be found. 

This system (|14|l as we shall demonstrate below possesses a so called coherent- 
temporal structure important for studying learning processes in biological neural 
networks. 

Assume for simplicity that all (3 - parameters are proportional to a small 
enough parameter e > 0, that is {/?} ~ {e/3} and consider first our flow 
at e = 0. It is easy to see that our model then possesses a closed orbit in the 
space of {x} and {y} - parameters, say a : S 1 — > M = R n x R m , satisfying the 
equation 

da/dr = -JVif(cr) (18) 

for all r G S 1 . Moreover, the Hamiltonian function H : M — > R in 115(1 is a 
conservation law of l|18|) . Take now e ^ 0; then one can state ( @|) that there 
exists a function H £ : M — > i?, such, that for some closed orbit a e : S 1 — > M this 
function 7J e : M — ► i? be a constant of motion (not a conservative quantity), 
that is for all small enough e > 

dH E (a E )/dt = 0(s 2 ) (19) 

as e — > 0. Then one can formulate the following proposition about the existence 
of a limiting cycle in our model at e > small enough. 
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Proposition. Let our model possess at small enough e > a smooth 
constant of motion H e : M — > R and a closed e-deformed orbit a e : S 1 — > i?. 
Moreover, at e = the constant of motion Ho : M — > i? is a hrst integral of the 
model in the neighborhood of the orbit ctq. Then a necessary condition for the 
existence of a limiting cycle at e > small enough is vanishing the following 
circular integral: 

j> (\7H Q (a ),gradV(a ))dt = 0. (20) 
s 1 

Having substituted expression 114fl into (|20p. one finds numerical constraints 
on the parameters locating our closed orbit Co : S 11 — > M in the phase space M. 
Thereby, we can localize this way possible coherent temporal patterns available 
in our neuron network under study. 

Using this approach let us consider the equation of motion on the vari- 
ables (x,y) £ R n+rn . The Lagrangian equation corresponding to potential (|14() . 
Hamiltonian (|15fl and matrix l|17|l can be represented as 

X W ( X ) = 0, (21) 



y ) \ y . 

where colums x = {a^i ...x n } T , y = {yi y m } T ', and matrix W = ( ^} ^} 

\ A 2 i>2 

Ai = J% + J n wJ m w T , A 2 = J„ + J m w T J n w, Bi = 3 2 n w + J n wJ m , B 2 = 
J^w + J m w T J n , w — {wik}- A solution to the matrix equation (|21|) can be 
represented as (x, y) T = acxp(iAt), with A G C being nontrivial only if the 
following determinant 

\-X 2 g + W\=0 (22) 

is equal to zero. 

Equation (1221) is one of the degree n + m subject to A 2 e C and determines 
2(n + m) eigen frequencies lu s = {±Wi, ±cj m+Tl } . In this case the solution 
gets the form 

Xs ) = a s exp(icij s t)+a* exp(— iuj s t). (23) 
Amplitudes a s = {a Sj i, a Syn + m } T must satisfy the matrix equation 

(~uo 2 s g + W) a s = 0. (24) 

If we take that amplitudes a s i — a* sl — 1 for any s € l,n + m solve l)24[l. we can 
get coefficients K S i of the distribution of amplitudes relative to any frequency 
Us- For this case the solution 123(1 can be represented as 

x S j = a s iK s j exp(za; s t)+a* X K S j exp(— iu> s t) 

= K Sti exp(iu s t)+K Sti exp(-iu s t), (25) 

y sj = K S j+ n exp(icj s t)+K Sl j+ n exp(— iuj s t). (26) 
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Here K Sy i, K s j +n are given constants depending on the frequencies uj s . Consider 
now the scalar product 

(VHo(ao), W(ffo)) = \xi + Yjuim j \ ~PiXi + P 2 4 + j + ■ 

m \ / 1 n 



n / y <~W,7~3 

3=1 / V 3=1 



m \ 



\ / 1 



3=1 / V 3=1 



Having substituted solution l|25|) into last expression and integrated it along the 
period T = 2ir/ui s we get a hyperplane which determines the parameters of our 
neural network model: 



Xi 



m \ / 3 1 " \ 

+ X>„3^ s ,„ +j I ( + J U^',. + !X 

K s , n+ i + £w a K s A [ -/3 3 if M+n + ^f3 4 Kl 1+n + ^f2^K sd+n 
i=i / \ " j=i 

Thus, as the index s changes from s = 1 to n + m, we can get in the general 
case n + m dimensional submanifolds determining parameters {Pi, f3 2 , ^3, Pi, 
@n ji Pn\}y a ^ wmcn the chosen oscillatory structure will persist for all / G 
thereby realizing a stable neural network and related with it the temporal patter 
under study. 
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